Cooperative automatic tracking

ABSTRACT

A system and method are provided for automatic cooperative object tracking using gain comparison of antenna pairs facing different directions. In cooperative object tracking, the object is associated with a radiation source, or beacon, that emits radiation that is detected by the tracking system. The present invention makes use of antennas that are not highly oriented antennas but are characterized by having a steep drop in their gain profiles at a particular angle of incidence of the radiation that they detect.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication No. 61/943,903, filed Feb. 24, 2014, titled “COOPERATIVEAUTOMATIC TRACKING”, the contents of which are hereby incorporated byreference in its entirety and are not admitted to be prior art withrespect to the present invention by the mention in this cross-referencesection.

BACKGROUND

This present invention is related to the field of orienting a pointingdevice at a beacon. The present invention is also related to the fieldof automatic cooperative object tracking (COP). The present invention isalso related to the field of automatic video recording using line ofsight (LOS) technology. This present invention is also related tomonopulse amplitude comparison based radar tracking,

SUMMARY OF THE INVENTION

In a system for cooperative tracking of an object, a pointer (alsoreferred to as a pointing device) is associated with a pan-tiltmechanism. Two (or more) antennas are associated with the pointer. Abeacon is associated with an object to be tracked. The beacon emits anidentifiable signal detectable by the antennas. A microcontrollercompares the gains of the two antennas causes the pan-tilt mechanism toturn in the direction of the antenna with the higher gain. In theinventive system, pairs of small solid state antennas are oriented insubstantially opposite directions.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 shows an illustration of a traditional antenna based trackingsystem featuring three antennas.

FIG. 2 is a schematic drawing explaining the operation of a traditionalradar based tracking system with two directional antennas.

FIG. 3 is a graph showing patch antenna gain versus signal's angle ofincidence.

FIG. 4 is schematic representation of an antenna arrangement of anautomated cooperative object tracking system in relation to a beaconaccording to a preferred embodiment of the present invention.

FIG. 5 is a schematic depiction of the angular distribution of gains ofan antenna pair according to a first preferred embodiment of the presentinvention.

FIG. 6 is a schematic depiction of the angular distribution of gains ofthree antennas according to another preferred embodiment of the presentinvention

FIG. 7 is a graph depicting the change of the angular location of abeacon orbiting a pointing device as detected according to the inventivemethod hereof and as deduced from GPS location detection data.

FIG. 8 is a perspective view of a pan and tilt tracking unit of anautomated cooperative object tracking system having two pairs ofdirectional solid state antennas according to a preferred embodiment ofthe present invention.

FIG. 9 is a schematic drawing of a wiring diagram of antennas and othercomponents of an automated cooperative object tracking system accordingto a preferred embodiment of the present invention.

FIG. 10 is a flow diagram of a method of orientation according to apreferred embodiment of the present invention.

FIG. 11 is a flow diagram of an automated editing and publishing methodof the footage recorded by the inventive system.

DETAILED DESCRIPTION

FIG. 1 shows an illustration of a traditional antenna based trackingsystem featuring three antennas. The direction of an incoming wavesignal can be determined using directional antennas. The traditionalMonopulse Amplitude Comparison (MAC) method is illustrated by FIG. 1 andFIG. 2. The application of this method requires the use of large, highlydirectional antennas, such as those depicted in FIG. 1. The antennasystem depicted in FIG. 1 is very large and is only transportable withthe use of a tractor-trailer due to its size. An example of the antennasystem depicted in FIG. 1 is described in the following publication: C.T. Nadovich, J. F. Aubin, D. R. Frey, An Instrumentation Radar Systemfor Use in Dynamic Signature Measurements (Jun. 1, 1992) (available at<http://www.microwavevision.com/sites/www.microwavevision.com/files/files/ORBIT-FR-InstrumentationRadarSystem-92-06-02-Nadovich_(—)0.pdf>).

FIG. 2 is a schematic drawing explaining the operation of a traditionalradar based tracking system with two directional antennas. FIG. 2 showshow two radar antennas are oriented such that the angle between theirmaximum gain vectors is only a few degrees. Gain values of a receivedsignal are compared between the two antennas of FIG. 2 and the angle Δθof the incoming signal from a source denoted as “target” can becalculated as the deviation from the orientation of the system of theantennas. By definition, this orientation is the line of symmetrybetween the two gain vector distributions (Beam 1 and Beam 2), which isdenoted as “Crossover axis” in FIG. 2. Thus, the crossover axis is atthe angle θ₀. For complete direction determination at least threeantennas are needed; these may be thought of being grouped into twocouples of antennas with orientation directions in two different(intersecting) planes. Highly directional antennas are too large to beused in small consumer electronic applications. An important feature ofthe MAC method is that the antennas are oriented in just slightlydifferent directions, θ_(s) in FIG. 2. Here the size of the angle θ_(s)is tied to the directionality (acceptance angle) of the antennas suchthat there is some overlap between the gain curves Beam 1 and Beam 2 ofthe antennas.

The present invention is an implementation of line of sight (“LOS”)technology for cooperative object tracking (“COT”). One importantapplications of the cooperative object tracking described herein is withautomated video recording of freely moving subjects. In such anapplication, a subject is equipped with a “remote device” that may becarried or worn that is also a radiation transmitter. The “remotedevice” may also be referred to herein as a beacon or target. To trackthe subject, an automated cooperative object tracking system preferablycomprises one or more receiver devices that receive the transmission ofthe beacon; the information contained therein is used to orient apointing device, such as a camera, at the beacon and, by implication, atthe subject. As will be described further herein, a multiplicity ofbeacons and/or a multiplicity of pointing devices may be used within theautomated cooperative object tracking system.

One of the disadvantages of using highly directional antennas for objecttracking is that if the object is significantly away from the directionof the antenna, the object is difficult to locate. In FIG. 2, the targetis located at a direction where the gain of at least one of the antennasis useful to detect it. Otherwise, the system shown in FIG. 2 would notbe able to locate the target easily, if at all. As will be describedbelow, the present invention is advantageous in that the object is never“lost” because the antennas employed are not narrowly oriented as theyare in FIG. 2.

According to a preferred embodiment hereof, the inventive system usestwo small (centimeter sized) solid state antennas facing very differentdirections (in some cases close to 180 degrees (i.e., close to oppositedirections) while in other cases the angle may be as small as 50degrees) to determine which of the pair receives the transmitted signalstronger. An example of the antenna that may be used is a patch antenna.A patch antenna is a type of radio antenna with a low profile mountableon a flat surface. Generally, patch antennas have a flat, rectangularsheet or “patch” of metal mounted over a larger sheet of metal called aground plane. A typical patch antenna gain pattern is shown in FIG. 3.FIG. 3 illustrates patch antenna gain vs. signal's angle of incidence.In FIG. 3, the antenna is in the center and it is oriented at θ=90degrees. The over 60 degree wide (3 dB drop-off) maximum gain width istoo broad for traditional MAC techniques. The inventive solution of thepresent invention only requires a steep gain slope which is acharacteristic for small patch antennas at about 90 degrees from thesurface normal. In FIG. 3 the steep gain slopes are at about +200degrees and at about +340 degrees (which also may be thought of as −20degrees). Also, patch antennas may be specially designed to have a sharpdrop off in gain at other particular angles. The antennas constitute apair (in other words, the antennas are paired), when the two antennasare directed in substantially different directions such that in thedirection of the half angle between the antennas' orientation theangular gradient of the gains of the antennas is maximum. Because of thetypical cylindrical symmetry of patch antennas, when the antennas arepaired, there is naturally no gain overlap except within and near to thecommon plane of the axes (common central plane) of the paired antennas.This is important in those instances when the beacon is not within thisplane or close to it. If this condition does not hold for a particularmake and/or model of antennas, one can add additional features to theirmounting to ensure that the no gain overlap substantially outside of thecommon central plane is fulfilled.

The invention described herein uses as an example radio frequency (RF)radiation being employed by the beacon. While this choice implies theuse of particular equipment (RF antennas, etc.) it is not implied thatusing other type of radiation is not within the scope of the presentinvention. For example, infrared (IR) and ultrasonic radiation sourcesand detectors are available and it is a matter of technological detailand choice as to which type of radiation is best to employ. Thewavelength choice within the RF band is of some significance; freeavailability of off the shelf equipment that does not require additionalcertification may be balanced by the desire of choosing wavelengths atwhich reflection effects (multipath errors) are minimal.

Additional mounting and shielding techniques may be used to cause thegain of the antenna to drop sharply at a particular angle. For example,mechanical barriers may be used such as extending the printed circuitboard (“PCB”) on which a patch antenna is mounted to block signals pasta desired angle (e.g., 90 degrees). In this arrangement, thedirectionality of the antenna is not important. That is to say, it isnot important in the same way as in the case of the example shown inFIG. 2 where a narrow maximum gain is necessary in the direction of theantenna. Here, on the contrary, the sharp drop-off of the gain along aparticular direction is the requirement. The important feature is thatthe antenna's gain drops sharply as the signal's angle of incidencepasses a certain angle.

FIG. 4 is schematic representation of an antenna arrangement of anautomated cooperative object tracking system in relation to a beaconaccording to a preferred embodiment of the present invention. FIG. 4shows a top view of a preferred embodiment of automated cooperativeobject tracking system 100. In the preferred embodiment of FIG. 4,automated cooperative object tracking system 100 is used to orientcamera 25 at a beacon 60. Beacon 60 is also a radiation transmitter.Automated cooperative tracking system 100 comprises a pair of patchantennas 20 and 30. In this example the patch antennas are mounted ontwo sides of panning unit 10 that can turn about an axis A (axis A isperpendicular to the plane of the drawing). Antennas 20 and 30 aremounted on PCBs 40 and 50, respectively. The sharp drop off of the gainof the antennas is enhanced by the shielding effect of extended PCBboards (an extended PCB is a PCB that has modifications of, for example,the size, thickness, coating, etc., to modify the gain profile of theattached antenna) on which each antenna is mounted, 45 and 55,respectively. Antenna 20 receives signal from beacon 60 at an angle θ1that is less than the drop off angle while antenna 30 receives thesignal from an angle θ2 that exceeds the drop off angle. As a result,antenna 20 receives a stronger signal than antenna 30 and a processingunit/microcontroller (not shown) compares the gain levels of thereceived signals by each antenna using, for example, the received signalstrength indicator (RSSI) method. Depending on which antenna had thehigher gain, the microcontroller will direct a turning mechanism (notshown) to turn panning unit 10 about axis A and rotate panning unit 10in an attempt to keep the gain levels of the two antennas the same,i.e., in a direction that minimizes the difference between the signalintensities detected by antenna 20 and antenna 30 (in counterclockwisedirection in the case of the example of FIG. 4).

FIG. 5 is a schematic depiction of the angular distribution of gains ofan antenna pair according to a preferred embodiment of the presentinvention. FIG. 5 illustrates the operation of the automated cooperativetracking system of FIG. 4 further. The orientation of the antennas istied to the orientation of the camera 25. For clarity, the antennas,PCBs, etc., are not shown in FIG. 5. The orientation of the optical axisof camera 25 is defined as 0 degrees. When the incoming radiationreaches the antennas of the automated cooperative tracking system at anangle that is less than 0 degrees, the left side antenna will have again (W_(left)) that is greater than the gain of the right side antenna(W_(right)), and the panning unit with camera 25 attached turns to theleft. In other words, the panning unit turns in the direction of theantenna with the higher gain. Note that the turning is described herewith the antennas being fixed on the panning mechanism. Thus, to turnthe camera toward the beacon, the antenna with the higher gain turnsaway from the beacon and the antenna with the lower gain turns towardthe beacon. The turning stops when the gains on both antennas are equal.Thus, the relationship between gain and the turning of the camera can begenerally stated as follows: If the left antenna gain is greater thanthe right antenna gain, then the camera is rotated to the left; if theright antenna gain is greater than the left antenna gain, then thecamera is rotated to the right.

The two antennas may have slightly different gain patterns. In such asituation, a calibration procedure solves this issue. The calibrationmay be done by placing a transmitter directly in front of the panningunit (at 0 degrees, by definition) and taking measurements of thereceived signal strength of each antenna. In an ideal situation, the twoantennas would receive the transmissions with equal strength when thetransmitter is directly in front of the panning unit. Practically,however, the strength of the received signal may be different betweenthe two antennas when the transmitter is directly in front of thepanning unit. Using the calibration values accounts for the differencesin gain patterns.

The antenna arrangement illustrated in FIG. 4 and FIG. 5 has thefollowing deficiency: if the beacon is in the space hemisphere that isbehind the camera (for example, close to 180 degrees) and there is asimilar crossover of the antenna gains at another degree, for example at180 degrees, then the panning device will turn the camera to thisdirection, which is not (and may be even the opposite of) the cameradirection. This is not a problem if the system is always used forfollowing targets that are in the front hemisphere with respect to thecamera. In a preferred embodiment of the present invention, this problemis resolved by using three or more antennas in the same plane. Thisarrangement is illustrated by FIG. 6.

FIG. 6 is a schematic depiction of the angular distribution of gains ofthree antennas according to another preferred embodiment of the presentinvention. In FIG. 6 the three antennas are shown schematically havingsimilar gain patterns that provide relatively high gains within a windowof about 150 degrees. The gain windows have sharp edges, where the gainfalls off and Antenna 1 and Antenna 2 are mounted such that theiroverlapping edges are at angle 0 degrees, which is the direction of thecamera. Antenna 3 is mounted such that it has relatively wide overlapsboth with Antenna 1 and Antenna 2.

To explain the operation of this antenna arrangement of FIG. 6, it isuseful to consider three beacon positions. Regarding Beacon 1, theorientation process is no different from that discussed with respect toFIG. 5 above. Since the signal on Antenna 1 is stronger than on Antenna2, a motor associated with the tracking unit will turn the tracking unitto the left and this process will continue until the signals on Antenna1 and Antenna 2 are equal.

Regarding Beacon 2, both Antenna 2 and Antenna 3 register signal fromBeacon 2. The tracking unit will turn to the right and this process willcontinue until Antenna 1 begins to register Beacon 2 and further untilthe signals on Antenna 1 and Antenna 2 are equal.

Regarding Beacon 3, Only Antenna 3 registers signal from Beacon 3. Thetracking unit will begin to turn in a preprogrammed direction (either tothe right, or to the left). Eventually, Beacon 3 will register onAntenna 1 (if the unit is turning to the left) or on Antenna 2 (if theunit is turning to the right). The turning will continue until Beacon 3registers on both Antenna 1 and Antenna 2 and further until the signalson Antenna 1 and Antenna 2 are equal.

In a slight modification, the tracking unit of the automated cooperativetracking system may be programmed to sense and register whether the gainof Antenna 3 increases or decreases after the tracking unit first startsturning. If the gain increases, the tracking unit reverses its turningdirection, but if the gain decreases, the tracking unit will keepturning in the initial direction. This modification may decrease thetime that elapses between first registering Beacon 3 and finally havingthe camera oriented at this beacon.

It is important to realize that the antenna arrangement of the automatedcooperative tracking system of the present invention is not limited tothree antennas. One may use four or more antennas that may have narrowergain windows, the advantage being that such antennas may have highergains. If there are N antennas numbered from 1 to N from left to rightand a beacon registers on antenna M, then, if M<½N, the tracking unitwill turn to right, but if M>½N, it will turn to left. In either casethe tracking unit will keep turning until both Antenna 1 and Antenna Nregister signal from the beacon and then until the signal registered isequal on both antennas. In the remainder of this disclosure we willrefer to such antenna systems as Paired Broad Gain Antenna systems, orPBGA systems, irrespective of the number (two or more) of antennas thatare actually in the system. It should be noted that to guide a trackingsystem both in pan and tilt directions, two PBGA systems must beemployed preferably arranged orthogonally (but orthogonal arrangement isnot necessary). Further, recognizing that in a single system Antenna 1and Antenna N constitute the antennas that make it a paired antennasystem, we will use the term “paired antennas” or “antenna pairs” todescribe these two antennas within a PBGA system.

FIG. 7 is a graph depicting the change of the angular location of abeacon orbiting a pointing device as detected according to the inventivemethod hereof and as deduced from GPS location detection data. The graphof FIG. 7 shows experimental data regarded as proof of concept for theautomated cooperative object tracking system and method of the presentinvention. To obtain the data shown in FIG. 7, a beacon was equippedwith both a radio transmitter and a GPS antenna. A pair of off-the-shelfTO-Link® TL-ANT2409A antennas were used as the receivers, in keepingwith the arrangement illustrated in FIG. 4. The antennas werestationary′ and the beacon was moved around to position it at differentangles with respect to the antennas. The angular positions of the beaconwere recorded using the GPS signal and are shown as black squares. Theapparent angular positions of the beacon were also measured using theinventive method and are shown as empty circles in FIG. 7. Consideringthat there were no measures taken to filter out electronic and othernoise, or to optimize the apparatus in any respect, the data show anacceptable degree of agreement between the angular positions deducedfrom GPS locating and those obtained using the inventive apparatus andmethod.

To orient a pointing device at a source of radiation in athree-dimensional space, one has to turn the pointing device about twoaxes that may be referred to as pan and tilt axes. The exampleillustrated in FIG. 4 may be regarded as either the pan portion or thetilt portion of such a pan and tilt apparatus. In one embodiment thetilt portion of the apparatus works essentially on the same principlesas described in conjunction with FIG. 4 and FIG. 5. However, there are afew more details useful to consider. To accomplish the task oforientation, one may use an apparatus comprising a panning unit mountedon a base, such as, for example, a tripod, and a tilting unit mounted onthe panning unit. If the antennas for panning are mounted on the sidesof the panning unit as described and shown above in FIG. 4, then theincoming signal from a moving radiation source that is at times at ahigher elevation and at other times is at a lower elevation arrives atthe antennas at varying angles. In such a situation, the characteristicsof the antennas may not be the same in all these directions. This maydefeat the calibration described above and may lead to inaccuracies inpointing. To avoid this, it is preferable to mount both the pan and thetilt antennas on the tilt portion of the orienting apparatus.

FIG. 8 is a perspective view of a pan-tilt mechanism of an automatedcooperative object tracking system having two pairs of directional solidstate antennas according to a preferred embodiment of the presentinvention. FIG. 8 illustrates one possible mounting arrangement of twopatch antenna pairs on a pan-tilt mechanism 200. Panning mechanism 210may be mounted on a tripod or other base via a rotatable shaft thatpermits the rotation of panning mechanism 210 around axis A shown as adashed-dotted line. Panning mechanism 210 is operationally coupled totilting mechanism 220. Tilting mechanism 220 is capable of rotationabout axis B, also shown as a dashed-dotted line. A portion of tiltingmechanism 220 is shaft 225 that serves as the mounting base for patchantenna 230, situated on the left end of shaft 225. Another patchantenna is similarly mounted on the right end of shaft 225 but is notshown in the drawing. Bases 240 and 250 hold patch antennas 245 and 255that provide tilt information for the microcontroller (not shown) ofpan-tilt mechanism 200. In keeping with the inventive method hereof anddescribed with the aid of FIG. 4, one of the patch antennas is mountedfacing up (antenna 255) and the other is mounted facing down (antenna245, shown using dashed lines). As noted above, more antennas may beused. Pan-tilt mechanism 200 is also equipped with omnidirectionalantenna 260 used to receive and to send radio signals. Screw 270 is usedto mount a camera or other pointing device on pan-tilt mechanism 200.

The RF tracking method of the present invention may a have a highsampling rate but noisy output. Therefore, filtering algorithms arepreferably employed to smooth the tracking data.

The signal emitted by beacon (e.g., beacon 60 FIG. 4) preferably carriesa code that makes it distinguishable from any otherwise similarbackground radiation.

FIG. 9 is a schematic drawing of a wiring diagram of antennas and othercomponents of an automated cooperative object tracking system accordingto a preferred embodiment of the present invention. In the preferredembodiment illustrated by FIG. 9, all patch antennas are connected to asingle amplifier and gain measurements are preferably carried outsequentially. In FIG. 9, automated cooperative object tracking system300 comprises four patch antennas 315, 325, 335, and 345, and onenon-directional antenna 350. All patch antennas are shown together withPCBs 310, 320, 330, and 340 on which they are mounted, respectively. Allantennas are connected directly or via the PCB to switch 360 thatconnects each antenna sequentially to microcontroller 370.Microcontroller 370 may preferably be of the type known as “RX MCU”. Asan example, the gains may be measured in the following order:left-facing antenna (antenna 345), right-facing antenna (antenna 335),up-facing antenna (antenna 325), and down-facing antenna (antenna 315,shown in dashed lines). This method is useful to minimize equipmentcost, but it assumes that measurements may be carried out sufficientlyfast. The system architecture shown and described in FIG. 9 may bemodified such that only some antennas are grouped and queried insequence. For example each PBGA system of a pan-tilt tracker may haveits own grouping of antennas in which the antennas are readsequentially.

Generally, if the beacon moves with a velocity v in a directionperpendicular to the pointing vector R, wherein vector R points from thelocation of the pointing device to the beacon, then the angular velocityof the pointer must be ω=v/R, where R is the length of vector R. If thebeacon emits signals in time intervals τ, then between two orientationreadings the pointer must turn an angle φ=ωτ. (The movement in thedirection perpendicular to the pointing vector R is the worst casescenario; however this calculation neglects the time required to measurethe gains and to translate such measurements into turning commands).Using the sequential gain measurement embodiment increases the effectivetime between readings by a factor of 4, to 4τ (or, if antenna 350requires equal time, 5τ). Whether this is acceptable for a particularpurpose depends on the expected velocities of the beacons and on theother technical characteristics of the system.

It is noted that techniques exist (e.g., RF ranging and IR intensitymeasurement) that can yield information concerning the distance betweenthe pointer and the beacon, i.e., the length R. Thus, R is assumed to beknown. It is also noted that the combined knowledge of the directionbetween the pointer and the beacon and of the distance between them issufficient to know their relative positions relative locations). In apreferred embodiment of the present invention, cameras that follow ortrack a beacon are preferably automatically turned on and off based onthe beacon's proximity to the camera, or to a particular camera if morethan one camera is used.

In a preferred embodiment of the present invention, the beacon ispreferably tracked using multiple pointing devices and each pointingdevice itself may also be used as a beacon. When the distance betweenany two of the pointers is known, it is possible to determine thelocations of all other pointers. This may be done during a setupprocedure before actual tracking and filming starts using the systemsand methods disclosed herein. Setup of such a system comprisesdetermining angles between the directions of any second and thirdpointers with respect to a first pointer and then using geometricalcalculations triangulation). Once the locations of all pointers areknown, the location of the beacon may be determined using orientationdata from any two of the pointers. Thus, if multiple pointers can trackthe same beacon, the location of the beacon may be determined usingmultiple sets of independent data. This opens the possibility of usingsuch data to determine the beacon's location with high certainty and toeliminate multipath effects. Eliminating multipath effects isparticularly important when such a system is used indoors where wallscan reflect radio waves.

In another preferred embodiment of the present invention, beacon 60 isalso equipped with one or more inertial measurement units (IMUs).Locating/orienting techniques are prone to errors (e.g., multipatherrors and blocked signals. These location errors can be reduced usingone or more IMUs on the beacon. The IMU measures the beacon'saccelerations, and the acceleration data can be transmitted to theorienting pan and tilt unit to be used by the microcontroller tosupplement the measurements of position of the beacon. For example, a3-axis accelerometer placed on the transmitter can send informationrelating to the magnitudes of accelerations experienced by the beacon(also called “motion indication values” [“MIVs”]). The filter settingson the pan and tilt unit may be adjusted in real time based on the MIVsreceived such that if an instantaneous location determination is farfrom the previous location indications, and the MIV is reporting lowvalues of accelerations, it is likely the new signal is the result of amulti-path reflection. Such a signal should either be ignored or heavilyfiltered. In another example, using a 6-axis IMU (meaning a 3-axisaccelerometer and 3-axis gyroscope), an estimate for distance traveledover a short period of time may be calculated. However the direction thebeacon moved would not be known. The transmitter associated with thebeacon could tell the pan-tilt unit that it moved a determined number ofmeters in the last second but not the direction of this movement. Themicrocontroller would then compare the new location estimate (based, forexample, on LOS reading and distance measurement) to the distanceestimate and the previous location and make sure that the locationreading is reasonable.

In another preferred embodiment of the invention a 9-axis IMU (3-axisaccelerometer, 3-axis gyroscope, 3-axis magnetometer) is used. With a9-axis IMU, an estimate for both distance and direction over a shortperiod of time can be calculated. A data fusion algorithm combines thetwo pieces of information, for example if the COT method detectsmovement of one meter and the IMU data predicts the movement was twometers over the same timeframe, the two values are averaged and thepointing device is instructed to point at the averaged value. Themicrocontroller uses such fused information for improving the overalllocation determination (to either detect or filter out errors caused bymultipath, or aid in extrapolation when LOS is not available due tosignal blocking).

In another preferred embodiment of the present invention, beacon 60preferably emits circularly polarized radiation and the patch antennasare preferably designed to detect such radiation. Radiation that doesnot reach the patch antennas directly, i.e., radiation that is reflectedfrom nearby objects, such as walls, will have lost the correctpolarization and will be detected less.

In another preferred embodiment of the present invention, when thesystem is employed for automated video recording, the microcontroller ofthe automated cooperative object tracking system may be programmed torecognize movement patterns that are characteristic of certainactivities. Such recognition may then be used to control recordingparameters. For example, when an exciting moment is about to occur (suchas when a surfer about to catch a wave), it would be desirable to havethe automatic recording device change to, for example, a high resolutionand faster frame rate to get better quality footage of the surfer andwave, and then revert back to a lower quality resolution and/or slowerframe rate after the exciting event. If the microcontroller isprogrammed to detect when an exciting event is about to happen, locationand velocity data can be used to detect an exciting event. For example,there are recognizable characteristics of IMU sensor data that can beused to detect that the surfer is likely about to catch a wave. Whensuch characteristics are detected, the microcontroller preferablytriggers the camera to record at a higher recording resolution and/orframe rate. One example of recognizable data would be aggressivepaddling motions detected by an IMU located on the surfers arm. Paddlinginto a wave by a surfer is much more aggressive when compared with otherpaddling scenarios and occurs only for a short period of time (about oneto five seconds).

Camera image stabilization algorithms that are optimized for stabilizinghand held video recording often do not work well (and are sometimesdetrimental) when the camera is being pointed by an electromechanicalcamera aiming system since the motion characteristics ofelectromechanical camera aiming systems often are much different thanthose of human cameramen. In a preferred embodiment of the presentinvention, the microcontroller of the pan and tilt mechanism implementsan image stabilization method that dampens high frequency vibrations butallows low frequency motions. This can be done using either mechanicalor digital methods, or using a combination of both.

In another preferred embodiment of the present invention, cooperativeobject tracking (COT) is combined with video recognition (VR). Forexample, the system starts tracking using a COT method (i.e., thesubject holds a tracking beacon and IR, RF, GPS, etc., are used to trackthe beacon). The VR software automatically learns to recognize thesubject that is being tracked and the system starts to track based on acombination of VR and COT. After some time, the system only needs VR totrack the subject. In a variation of this method, the subject spends acertain amount of time using the beacon and “teaching” the system tooperate based on VR, then the use of the beacon is discontinued. Oneadvantage of this method is that the beacon may be used less requiringless battery power permitting use of a smaller size beacon. A furtheradvantage of this method is that the subject can become free from havingto carry the beacon. In another embodiment, after the system “learns” torecognize the subject, the system “learns” to recognize other subjectsof the same “class”. Here “class” signifies, for example, a group ofpeople engaged in similar activity (e.g., riding bicycles, surfing,etc.), or wearing similar clothing (e.g., uniforms as on a soccer team).Systems using class tracking may include a multiplicity of trackingcameras in a network that would follow members of the class based ontheir location (e.g., being within the soccer field, in the water,etc.), or other criteria (e.g., velocity, distance from the camera,etc.). Thus, the network of tracking cameras uses cooperative objecttracking to start, but eventually uses the network's improved pixeltracking to track objects or events that are not being tracked withcooperative object tracking with the system.

FIG. 10 is a flow diagram of a method of orientation according to apreferred embodiment of the present invention. Regarding such method, itis assumed that the tracking system is oriented nearly at the objectsuch that the radiation signal emitted by the beacon is registered byboth antennas of each antenna pair mounted on orthogonal planes. Abeacon associated with the object to be tracked periodically transmits asignal in step 410. The signal preferably incorporates a uniqueidentifier of the beacon. The signal is preferably a radio wave, but itmay be electromagnetic radiation from a different part of theelectromagnetic spectrum (e.g., infrared light) or a sound wave (e.g.,ultrasound). The signal transmitted from the beacon is detected byappropriate antennas associated with a pointing device, such as a videocamera, having a pan-tilt mechanism, in step 420. The antennas comprisetwo orthogonally arranged Paired Broad Gain Antenna (PBGA) systems. Thegain of each paired antenna is determined in step 430. For each antennapair the gain difference is determined algebraically, i.e., both thesign (i.e., which antenna has the higher gain) and the magnitude of thegain difference are determined in step 440. In step 450, the systemdetermines if the gain difference is nearly zero or not. If the gaindifference is nearly zero, no action is taken (step 460). If the gaindifference is not nearly zero, then the method proceeds to step 470. Instep 470, the pan-tilt mechanism of the pointing device turns in thedirection of the higher gain of each antenna pair. That is, the gaindifference detected within the panning plane causes the mechanism to panif that difference is not nearly zero and the gain difference detectedin the tilt plane causes the mechanism to tilt if that difference is notnearly zero. The definition of “nearly zero” is equivalent to defining adeadband that is useful to avoid jittery reaction to minute changes inthe detected antenna gains. Such minute changes may be due to minutechanges in the position of the object tracked or may be due to systemnoise. In step 480, turning velocity is controlled in part based on themagnitude of the gain difference detected between each pair of antennas.This is based on the recognition that when there is a higher differencein gain, the object orientation is farther from the current orientationof the pointer.

When orienting a camera at a moving object, like a person carrying abeacon, commands for orienting the camera have to be provided to theorienting mechanism with sufficient frequency. This frequency is,however still relatively low, in the range of three to 10 Hz in mostcases. Also, a beacon associated with a person (for example, as a deviceattached to an armband) may move in such a way that the radiationintensity emitted in the direction of the receiving antennas may changerapidly. To avoid errors associated with such changes between themeasurements of the gains of the antenna pair, one may want to domeasurements in very quick succession, for example at a rate of 100 Hz.Using this as an example, one would generate 50 gain difference readingsper second, but orientation commands to the pan-tilt mechanism thatorient the pointing device/camera need to be provided only at a rate ofabout 5 Hz, for example. That would allow for an averaging of 10 gaindifference readings before each turning command and a correspondingimprovement of signal-to-noise ratio. Alternatively, Kalman filteringmay also be employed.

In another preferred embodiment of the present invention, the trackingapparatus is a part of a system that comprises other similar apparatuses(automated cameramen) and also a human cameraman, or more generally, ahuman user in control of the system. The human user may or may not be apart of the action that is being filmed. The human user can determinewhen the action is worth recording or photographing. A problem with anautomated cameramen is that they do not have the same intelligence as ahuman to determine when to start or stop filming. For example, automatedcameramen will not inherently respond to a director yelling “action”.The one or more automated cameramen in the present embodiment start orstop recording (or take photos) based on the remote control of a human.The remote control is activated when the human user is a cameraman anduses their camera such that the automated cameramen mimics what thehuman cameraman is doing. For example, when the human cameraman beginsrecording, a signal is sent to all the automated cameramen which alsostart recording.

In a yet another preferred embodiment of the present invention multipletracking devices are interconnected to form a system wherein eachtracking device utilizes data from all devices in a machine learningprotocol to learn from each other so that tracking uses location andbiometric data compared with pixel tracking and all the other types oftracking to further optimize the tracking and video recording process.As a result, a network of tracking devices will act as an array of eyesand ears that automatically senses and alerts to problems. Examples are:(a) a camera “sees” in its peripheral that a car accident occurred andautomatically dials 911; (b) a camera that is part of a municipalnetwork “sees” a police officer in trouble and alerts the policedepartment; (c) a camera is following a surfer but senses a drowningvictim in the background, or even recognizes whale breaching, and alertsthe network.

In yet another preferred embodiment of the present invention, thetracking apparatus is used to recognize events. For example, a sportingevent can be determined based on the types of motions detected (e.g.,youth soccer). The inventive system of tracking devices may alsopreferably connect with a cellular company (such as Verizon) to detecttheir customers in the vicinity of an event based on user's cell phonelocation data. The company can reasonably assume that those customersare also interested in that type of event (e.g., youth soccer). Thenanyone with a Verizon phone in the vicinity may be flagged as beinginterested in youth soccer and targeted for related advertising.

During the cooperative object tracking, video recognition algorithms mayalso be used for collecting additional customer data.

FIG. 11 is a flow diagram of an automated editing and publishing methodof the footage recorded by the inventive system. A problem of uploadingvideo footage to a remotely located editing service is that videofootage files are large and time consuming to transfer. An inventive wayto solve this problem is illustrated using FIG. 11. In this method theuser films, using the inventive system and method hereof, footage inhigh resolution in step 500. The footage is saved on the user's devicein step 510. The user's device creates a low resolution version of thefootage in step 520, and uploads it to a remotely located server(computer) of an editing service in step 530. The editing service thenedits the low resolution footage into “appealing video clips” in step540. The editing software that the editing service uses then creates aset of specific editing instructions that are to be implemented on thehigh resolution version on the user's device in step 550. Theinstructions are sent to the user and are implemented on the user'sdevice that has editing software capable of following the editinginstructions generated in step 550 and replicates the appealing videoclips(s) on the user's device using the high resolution video files(step 560). The result may be reviewed by the user in step 570. The userdecides whether the edit is “good” or not in step 580. If not, theediting process may be repeated by returning to step 540. At this step,the user may provide editing suggestions, instructions, and the like tomake the second round of editing improved. If the user is satisfied withthe edit, the user uses the high resolution edited footage in step 590.For example, the user may upload the edited clip to a server for viewingby others.

One benefit of the method of FIG. 11 is that large, high resolutionvideo files never needed to be uploaded for editing. The remotelylocated video editing service may be an automated service or a serviceperformed by humans. Step 580 is optional as noted by the dashed lineconnecting step 570 with step 590. In a preferred embodiment, suchautomated editing service collects feedback data to improve its editingcapabilities. As more people use the service and provide feedback, thequality of the editing service improves. The feedback could be based onusers modifying the edited video clips on their own computers software.Data on the modifications which the user made is sent as feedback to theediting service to improve their editing algorithms.

Applicant hereby incorporates by reference in their entirety thefollowing co-owned patent applications which may assist in understandingthe present invention: U.S. patent application Ser. No. 13/801,336,titled “System and Method for Video Recording and Webcasting SportingEvents”, PCT International Patent Application No. PCT/US2013/041187,titled “High Quality Video Sharing Systems”, PCT International PatentApplication No. PCT/US2013/070903, titled “Automatic Cameraman,Automatic Recording System and Automatic Recording Network”, and U.S.patent application Ser. No. 14/600,177, titled “Neural Network for VideoEditing”.

Different embodiments, features and methods of the invention aredescribed with the aid of the figures, however the particular describedembodiments, features and methods should not be construed as being theonly ones that constitute the practice of the invention and thedescribed embodiments, features and methods are in no way substitutesfor the broadest interpretation of the invention as claimed.

What is claimed is: 1) A system for automated cooperative tracking of anobject, said system comprising: a) a pointer associated with a pan-tiltmechanism, wherein said pan-tilt mechanism has panning and tilting partscapable of turning around a panning axis and around a tilting axis,respectively, and wherein said pointer has a pointing direction withrespect to said panning and tilting parts; b) a first antenna pairassociated with said pan-tilt mechanism, said first antenna paircomprising a first broad gain antenna and a second broad gain antenna,wherein said first and second broad gain antennas are pointing insubstantially different directions; c) a beacon associated with theobject, wherein said beacon emits an identifiable signal and said firstand second broad gain antennas detect the identifiable signal; and d) amicrocontroller that compares the gains of said first and second broadgain antennas and causes said pan-tilt mechanism to turn in thedirection of the broad gain antenna with the higher gain. 2) The systemfor automated cooperative tracking of an object of claim 1, wherein saidfirst and second broad gain antennas are characterized by having theirgains decrease rapidly around and crossing over substantially at thepointing direction of the pointer within a plane perpendicular to thepanning axis. 3) The system for automated cooperative tracking of anobject of claim 2, wherein said first and second broad gain antennashave substantially no gain overlap outside of said plane. 4) The systemfor automated cooperative tracking of an object of claim 2, furthercomprising a second antenna pair associated with said pan-tilt mechanismcomprising a third broad gain antenna and a fourth broad gain antenna,wherein said third and fourth broad gain antennas are characterized byhaving their gains decrease rapidly around and crossing oversubstantially at the pointing direction of the pointer within a planeperpendicular to the tilting axis and having no gain overlapsubstantially outside of said plane. 5) The system for automatedcooperative tracking of an object of claim 2, wherein said pointer alsocomprises at least one additional broad gain antenna, wherein said atleast one additional broad gain antenna points to a direction within theplane perpendicular to the panning axis of said pan-tilt mechanism. 6)The system for automated cooperative tracking of an object of claim 4,wherein said pointer further comprises at least one additional broadgain antenna, wherein said at least one additional broad gain antennapoints to a direction within the plane perpendicular to the tilting axisof said pan-tilt mechanism. 7) The system for automated cooperativetracking of an object of claim 6, wherein said beacon further comprisesat least one inertial measurement unit and a transmitter to broadcastdata obtained by said at least one inertial measurement unit to saidpointer. 8) A method of automated cooperative tracking of an object,said method comprising the steps of: a) emitting a signal from thelocation of the object to be tracked; b) providing a tracking mechanismequipped with at least a first and a second broad gain antennas mountedon the tracking mechanism such that the first and second broad gainantennas constitute a first set of paired antennas; c) detecting signalsemitted from the object to be tracked using the broad gain antennas,wherein the detected signal on each of the first and second broad gainantennas is characterized by an intensity; d) turning the trackingmechanism such that at least one of the paired antennas detects a signalwith a non-zero intensity; e) comparing the signal intensities detectedby the first set of paired antennas and turning the tracking mechanismin the direction of the first antenna if it is detecting higher signalintensity than the second antenna and turning the tracking mechanism inthe direction of the second antenna if it is detecting higher signalintensity than the first antenna; f) stopping the tracking mechanism ifthe signal intensities detected by the first antenna and by the secondantenna are substantially the same and not zero. 9) The method ofautomated cooperative tracking of an object of claim 8, furthercomprising the steps of: a) providing a second set of paired antennas,wherein the second set of paired antennas is mounted substantiallyorthogonally to the first set of paired antennas; b) turning thetracking mechanism such that at least one of the second set of pairedantennas detects a signal with a non-zero intensity; c) comparing thesignal intensities detected by the second set of paired antennas andturning the tracking mechanism in the direction of the antenna that isdetecting higher signal intensity; d) stopping the tracking mechanism ifthe signal intensities detected by each of the second set of pairedantennas are substantially the same and not zero. 10) The method ofautomated cooperative tracking of an object of claim 9, also comprisingthe steps of: a) providing additional unpaired antennas pointing insubstantially different directions than the paired antennas; and b)turning the tracking mechanism such that the signal intensities detectedby the additional unpaired antennas is substantially zero. 11) Themethod of automated cooperative tracking of an object of claim 8,further comprising the step of controlling the turning velocity of thetracking mechanism at least in part based on the difference of signalintensities detected by the first and second antennas. 12) The method ofautomated cooperative tracking of an object of claim 8, furthercomprising the step of averaging the signal intensities detected by thefirst and second antennas over a time period of about 200 ms. 13) Themethod of automated cooperative tracking of an object of claim 8,further comprising the steps of: a) detecting movements of the objectusing inertial measurement units; b) calculating expected directions forthe tracking mechanism based on the detected movements of the object.14) A method of automated tracking and filming of an object, said methodcomprising: a) providing a beacon associated with the object, whereinthe beacon emits a signal; b) providing a first tracking mechanismequipped with a camera at a first location, wherein the first trackingmechanism comprises a first wide angle antenna and a second wide angleantenna, and wherein the first and second wide angle antennas detectsignal emitted by the beacon and wherein the first and second wide angleantennas are paired, and the first tracking mechanism also comprises amicrocontroller that compares intensities of signal detected by thefirst and second wide angle antennas and causes the tracking mechanismto turn such that any difference between the detected signal intensitiesis diminished; c) recording and saving the pointing direction of thecamera when the difference between the detected signal intensities issubstantially zero; and d) recording footage of the object. 15) Themethod of automated tracking and filming of an object of claim 14further comprising the steps of: a) providing a multiplicity of trackingmechanisms, located at a multiplicity of different locations, whereineach of the multiplicity of tracking mechanisms are substantiallyidentical to the first tracking mechanism, including having a camera,and wherein the distances and directions between the multiplicity oflocations are known; b) operating the multiplicity of trackingmechanisms substantially similar to the first tracking mechanismincluding recording and saving the directions of the cameras when thedifferences between the detected signal intensities on their pairedantennas are substantially zero; c) using the directions of the camerascombined with the known distances and directions between themultiplicity of locations to calculate locations of the object, and d)recording footage of the object with the cameras. 16) The method ofautomated tracking and filming of an object of claim 14, furthercomprising the steps of: a) using circularly polarized radiation tocarry the signal emitted by the beacon; b) detecting the circularlypolarized radiation using patch antennas designed to preferably detectcircularly polarized radiation. 17) The method of automated tracking andfilming of an object of claim 15, further comprising the step ofstarting and stopping video recording and taking still photographs usinga remote control. 18) The method of automated tracking and filming of anobject of claim 15, further comprising the steps of: a) using imagerecognition software to analyze the recorded footage of the object; b)using machine learning strategies to improve image recognition of theobject; and c) tracking the object using the improved image recognition.